🕷️️ Job Radar • SCRAPING

upwork.com 2026-04-30 🟢

🔹 Build Scalable Real Estate Scraping Framework
👤 Client: 🇳🇱 Netherlands Member since 2021-12-04
💰 Price: $800
🚩 Problem: Scrape structured real estate data from multiple websites with a reusable and scalable system.
📦 Existing: Not specified

Specifications:

[Target] Scrape real estate data from hundreds of websites.
[Method] Use Python, Scrapy + Playwright for scraping.
[UI/UX] Not applicable (backend project).
[Stack] Python, Scrapy, Playwright, YAML/JSON config files.
[Security] Basic anti-bot handling including throttling and retries.
[Format] Standardized JSON output with structured data.

Workflow:

Set up a reusable scraping framework using Python, Scrapy, and Playwright.
Develop base spider with shared core logic.
Create one YAML/JSON config file per website for easy site addition.
Implement pagination handling and nested property detail page support.
Ensure dynamic content and JS-rendered websites are supported.
Add basic anti-bot mechanisms including throttling, headers, and retries.
Develop logging for errors such as missing selectors and blocked pages.
Implement retry logic to handle partial failures.
Extract amenities/features from description text using simple keyword/synonym mapping.
Support English and Spanish websites with standardized feature keys.
Design the system for scalability, capable of handling 500+ sites initially and adding ~10 new sites per day.
Ensure the framework is user-friendly for non-expert freelancers.